Enhanced cursor control using interface devices

ABSTRACT

An interface device and method for providing enhanced cursor control with force feedback. A force feedback interface device includes a manipulandum, such as a mouse, that is moveable in a local workspace. The device is coupled to a host computer that displays a cursor in a graphical environment, such as a GUI, on a display screen. An interior region and a border region in the local workspace is defined. One mapping of device movement to cursor movement is used for the interior region, and a different mapping is used for the border region. Mapping methods include ballistics, absolute, linear, rate control, and variable absolute. Rate control embodiments can be single axis or dual axis. In one embodiment, when the mouse moves from the interior region to the border region, the mapping providing the greater cursor velocity is used to better conserve device workspace in the direction of travel and to decrease any sense of mapping mode change to the user. Other features include an autocentering function for reducing offset between local and host frames.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of pending prior U.S. applicationSer. No. 09/565,574, filed on May 4, 2000, now U.S. Pat. No. 6,292,174in the name of Jeffrey R. Mallett, et al., which 1) is acontinuation-in-part of prior U.S. application Ser. No. 08/924,462, nowU.S. Pat. No. 6,252,579, filed on Aug. 23, 1997, and 2) claims priorityto U.S. Provisional Application No. 60/133,227, filed May 7, 1999, allof which are incorporated herein by reference in their entireties.

This invention was made with government support under Contract NumberF41624-96-C-6029, awarded by the Department of Defense. The governmenthas certain rights in this invention.

BACKGROUND OF THE INVENTION

The present invention relates generally to interface devices forallowing humans to interface with computer systems, and moreparticularly to computer interface devices that allow the user toprovide input to computer systems and provide force feedback to theuser.

Graphical environments are commonly displayed on computer systems. Onevisual environment that is particularly common is a graphical userinterface (GUI). The user typically moves a displayed, user-controlledgraphical object, such as a cursor, across a computer screen and ontoother displayed graphical objects or predefined screen regions, and theninputs a command to execute a given selection or operation. The objectsor regions (“targets”) can include, for example, icons, windows,pull-down menus, buttons, and scroll bars. Most GUI's are currently2-dimensional as displayed on a computer screen; however, threedimensional (3-D) GUI's that present simulated 3-D environments on a 2-Dscreen can also be provided. Other programs or environments that mayprovide user-controlled graphical objects such as a cursor or a “view”controlled by the user include graphical “web pages” or otherenvironments offered on the World Wide Web of the Internet, CADprograms, games, virtual reality simulations, etc.

The user interaction with and manipulation of the computer environmentis achieved using any of a variety of types of human-computer interfacedevices that are connected to the computer system controlling thedisplayed environment. In most systems, the computer updates theenvironment in response to the user's manipulation of auser-manipulatable physical object (“user object”) that is included inthe interface device, such as a mouse, joystick, etc.

A computer mouse is a common user object used to interact with a GUI orother graphical environment. A mouse is typically used as a positioncontrol device in which displacement of the mouse in a planar workspace(e.g. on a mouse pad) is directly correlated to displacement of theuser-controlled graphical object, such as a cursor, displayed on thescreen. This displacement correlation may not be a one-to-onecorrespondence, since the cursor position may be scaled according to aconstant mapping from the mouse position e.g., the mouse may be moved adistance of one inch on a mouse pad which causes the controlled cursorto move four inches across the screen. In most cases, small movements ofthe mouse are scaled to large motions of the cursor on the screen toallow the user to easily point to targets in all areas of the screen.The user can typically change the scaling or “pointer speed” of a cursorto a desired level, which is the ratio or scaling factor of cursormovement to mouse movement, using menus provided in the operating systemor application program.

Force feedback interface devices, such as force feedback mice, allow auser to experience forces on the manipulated user object based oninteractions and events within the displayed graphical environment.Typically, computer-controlled motors or other actuators are used tooutput forces on the user object in provided degrees of freedom tosimulate various sensations, such as an obstruction force when movingthe cursor into a wall, a damping force to resist motion of the cursor,and a spring force to bias the cursor to move back toward a startingposition of the spring.

The scaled cursor movement in a GUI works well for coarse cursor motion,which is the broad, sweeping motion of the cursor that brings the cursorfrom one global area on the screen to another. Accuracy of cursor motionis not critical for coarse motion, but speed of the cursor is. For suchtasks, it is valuable for the cursor to move a large distance with smallmotions of the physical mouse. However, a problem occurs when the userwishes to move the cursor a short distance or in small increments (“finepositioning”). For tasks in which accurate positioning of the cursor isneeded, such as target acquisition tasks, the large scaling of mousemovement to cursor movement is inadequate or even harmful. Certaintarget acquisition tasks where the targets are very small can beparticularly challenging even if the mapping between the cursor and themouse is reasonable for most other cursor motion activities. For suchsituations, a scaling that causes large motions of the cursor for smallmotions of the mouse may make a target acquisition task physicallyimpossible for the user.

Mouse “ballistics” or “ballistic tracking” is typically used toalleviate the scaling problem for fine positioning of the cursor.Ballistics refers to the technique of varying the scaling between motionof a physical mouse and motion of a displayed cursor depending upon thevelocity of the mouse in its workspace. The assumption is that if theuser is moving the mouse very quickly, the user is likely performing a“coarse motion” task on the screen, and therefore the mouse driverscales small motions of the mouse to large motions of the cursor.Conversely, if the user is moving the mouse very slowly, the user islikely performing a fine positioning task on the screen, and the mousedriver scales small motions of the mouse to small motions of the cursor.Such a variable scaling technique is disclosed in U.S. Pat. No.4,734,685 of Watanabe and U.S. Pat. No. 5,195,179 of Tokunaga.

Many algorithms can be used for mouse ballistics. The simplest method isto designate a threshold velocity such that if the mouse is movingfaster than the threshold velocity, a large scaling of cursor positionis made so that small motions of the mouse cause large motions of thecursor; and if the mouse is moving slower than the threshold velocity, asmaller scaling is made so that small motions of the mouse cause smallmotions of the cursor. A more sophisticated and more common method is togradually change the scaling in accordance with mouse velocity usingseveral velocity thresholds or a continuous (linear or nonlinear)function. The “mapping” of the cursor to the mouse is the method oftranslating the mouse position in its workspace to a cursor position onthe display screen and may involve ballistics or other algorithms andscale factors.

Mouse ballistics and other mappings may cause difficulty in certainfixed-workspace force feedback mouse implementations. Using ballistics,moving the mouse in one direction quickly and then moving it back in theother direction slowly may create a situation where the physical mousehas returned to its starting position but the cursor is positioned faraway from its starting position. This illustrates that the frame of thecursor and the frame of the mouse have shifted or become offset. If thisoffset becomes too large, the user may not be able to reach some partsof the screen within the range of motion of the mouse. In a typical,open-workspace mouse, the offset is corrected through a process called“indexing.” Indexing is achieved in a typical mouse by lifting the mouseoff the table and repositioning it after the mouse has hit a limit,while the cursor remains fixed in position. This reduces the offsetbetween the mouse and the cursor frames to a smaller, more comfortableoffset. However, some types of force feedback mice may have a fixed,limited workspace due to cost and technological constraints and may notbe able to be lifted off the table and repositioned. In addition, themouse hitting a physical limit to its workspace is disconcerting for auser expecting realistic force feedback. Thus, traditional indexing (orits equivalent) may not be practical. However, since ballistics needsindexing to restore the frame offsets, and since ballistics and indexingare both traditional mouse techniques that conflict with typical forcefeedback functionality, a solution is needed that reconciles both theballistics and the indexing problem in force feedback interface devices.

SUMMARY OF THE INVENTION

The present invention is directed to a force feedback interface whichallows enhanced cursor control using fixed-workspace force feedbackdevices. Various embodiments are presented which compensate for anyproblems caused by offsets in mouse and cursor movement frames.

An interface device and method for providing enhanced cursor controlwith force feedback is described. A force feedback interface deviceincludes a manipulandum, such as a mouse, that is moveable in a local,fixed workspace. The device is coupled to a host computer that displaysa cursor in a graphical environment, such as a GUI, on a display screen.An interior region and a border region in the local workspace isdefined. One mapping of device movement to cursor movement is used forthe interior region, and a different mapping is used for the borderregion. Mapping methods include ballistics, absolute, linear, ratecontrol, and variable absolute methods. The rate control methods can beused for a single axis or for two axes.

In one embodiment, when the mouse moves from the interior region to theborder region, the mapping providing the greater cursor velocity is usedto decrease any sense of mapping mode change to the user. In a differentembodiment, the force feedback device performs auto-centering of themanipulandum in its workspace using the actuators of the device todecrease an offset between the local frame and the screen frame.

The methods and apparatus of the present invention advantageouslyprovides enhanced control over a cursor in a graphical environment for akinesthetic force feedback mouse with a limited workspace, while notcompromising the fidelity or expected feel of force feedback sensationsbased on motion of the mouse or other user object. The indexing featuresof the present invention allow the user to control the cursor even whena large offset exists between the mouse and cursor positions in theirrespective frames, allows the user to reduce this offset, andsubstantially reduces the user's undesired experience of any hard,physical stops when the mouse reaches a physical limit in its workspace.

These and other advantages of the present invention will become apparentto those skilled in the art upon a reading of the followingspecification of the invention and a study of the several figures of thedrawing.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a force feedback mouse embodimentsuitable for use with the present invention;

FIG. 2 is a perspective view of a mechanical system for use with theinterface device of FIG. 1;

FIGS. 3a and 3 b are top plan views of the mechanical system of FIG. 2showing the limits to motion of the mouse;

FIG. 4 is a block diagram illustrating the interface device and hostcomputer suitable for use with the present invention;

FIG. 5 is a diagrammatic illustration of the screen frame and deviceframe of the mouse device;

FIG. 6 is a flow diagram illustrating a method of providing a hybridmapping method of the present invention; and

FIG. 7 is an illustration of a displayed interface for selectingfeatures of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1 is a perspective view of a force feedback mouse interface system10 of the present invention capable of providing input to a hostcomputer based on the user's manipulation of the mouse and capable ofproviding force feedback to the user of the mouse system based on eventsoccurring in a program implemented by the host computer. Mouse system 10includes a mouse or “puck” 12, an interface 14, and a host computer 18.It should be noted that the term “mouse” as used herein, indicates anobject 12 generally shaped to be grasped or contacted from above andmoved within a substantially planar workspace (and additional degrees offreedom if available). Typically, a mouse is a smooth or angular shapedcompact unit that snugly fits under a user's hand, fingers, and/or palm,but may be shaped otherwise in other embodiments.

Mouse 12 is an object that is preferably grasped or gripped andmanipulated by a user. By “grasp,” it is meant that users may releasablyengage a portion of the object in some fashion, such as by hand, withtheir fingertips, etc.; an example of a user's hand is shown as dashedline 16. In addition, mouse 12 preferably includes one or more buttons15 to allow the user to provide additional commands to the computersystem. It will be appreciated that a great number of other types ofuser manipulable objects (“user objects” or “physical objects”) andmechanisms suitable therefor can be used, such as a sphere (e.g. a trackball), a puck, a joystick, cubical- or other-shaped hand grips, areceptacle for receiving a finger or a stylus, a flat planar surfacelike a plastic card having a rubberized, contoured, and/or bumpysurface, or other objects.

Interface 14 interfaces mechanical and electrical input and outputbetween the mouse 12 and host computer 18 implementing the applicationprogram, such as a GUI, simulation or game environment. Interface 14provides multiple degrees of freedom to mouse 12; in the preferredembodiment, two planar degrees of freedom are provided to the mouse, asshown by arrows 22. In other embodiments, greater or fewer degrees offreedom can be provided, as well as rotary degrees of freedom. For manyapplications, mouse 12 need only be moved in a very small workspacearea, shown as dashed line 24 in FIG. 1 as an example. In a preferredembodiment, the user manipulates mouse 12 in a planar workspace and theposition of mouse 12 is translated into a form suitable forinterpretation by position sensors of the interface 14.

The mouse 12 and interface 14 preferably provide kinesthetic forces tothe mouse 12 and thus to the user grasping or otherwise contacting themouse. That is, forces are provided on the user manipulatable object inthe planar degrees of freedom of the mouse, such as resistance tomotion, jolts, textures, etc. In the preferred embodiment, the mouse 12has a limited workspace and will hit the hard limits when moved to theworkspace extremes. The limited workspace is due to a mechanical linkageproviding the forces on the mouse, as described below with respect toFIG. 2.

The electronic portion of interface 14 may couple the mechanical portionof the interface to the host computer 18. The electronic portion ispreferably included within the housing 26 of the interface 14 or,alternatively, the electronic portion may be included in host computer18 or as a separate unit with its own housing. A suitable embodiment ofthe electrical portion of interface 14 is described in detail withreference to FIG. 4.

The interface 14 can be coupled to the computer 18 by a bus 17, whichcommunicates signals between interface 14 and computer 18 and also, inthe preferred embodiment, provides power to the interface 14 (e.g. whenbus 17 includes a USB interface). In other embodiments, signals can besent between interface 14 and computer 18 by wirelesstransmission/reception.

Host computer 18 is preferably a personal computer or workstation, suchas an IBM-PC compatible computer or Macintosh personal computer, or aSUN or Silicon Graphics workstation. For example, the computer 18 canoperate under the Windows™ or MS-DOS operating system in conformancewith an IBM PC AT standard. Alternatively, host computer system 18 canbe one of a variety of home video game systems commonly connected to atelevision set, such as systems available from Nintendo, Sega, or Sony.In other embodiments, home computer system 18 can be a “set top box”which can be used, for example, to provide interactive televisionfunctions to users, or a “network-” or “Internet-computer” which allowsusers to interact with a local or global network using standardconnections and protocols such as used for the Internet and World WideWeb. Host computer preferably includes a host microprocessor, randomaccess memory (RAM), read only memory (ROM), input/output (I/O)circuitry, and other components of computers well-known to those skilledin the art.

Host computer 18 preferably implements a host application program withwhich a user is interacting via mouse 12 and other peripherals, ifappropriate, and which can include force feedback functionality. Forexample, the host application program can be a simulation, video game,Web page or browser that implements HTML, VRML, or other instructions,scientific analysis program, virtual reality training program orapplication, or other application program that utilizes input of mouse12 and outputs force feedback commands to the mouse 12. Herein, forsimplicity, operating systems such as Windows™, MS-DOS, MacOS, Unix,etc. are also referred to as “application programs.” In one preferredembodiment, an application program utilizes a graphical user interface(GUI) to present options to a user and receive input from the user.Herein, computer 18 may be referred as displaying “graphical objects” or“computer objects.” These objects are not physical objects, but arelogical software unit collections of data and/or procedures that may bedisplayed as images by computer 18 on display screen 20, as is wellknown to those skilled in the art. Display device 20 can be included inhost computer 18 and can be a standard display screen (LCD, CRT, etc.),3-D goggles, or any other visual output device.

There are two primary “control paradigms” of operation for mouse system10: position control and rate control. Position control is the moretypical control paradigm for mouse and similar controllers, and refersto a mapping of mouse 12 in which displacement of the mouse in physicalspace directly dictates displacement of a graphical object. The mappingcan have an arbitrary scale factor, but the fundamental relation betweenmouse displacements and graphical object displacements should bepresent. Under a position control mapping, the computer object does notmove unless the user object is in motion. Position control is a popularmapping for applications such as graphical user interfaces (GUI's) ormedical procedure simulations. Position control force feedback roughlycorresponds to forces which would be perceived directly by the user,i.e., they are “user-centric” forces.

As shown in FIG. 1, a “display frame” 28 is provided with the displayscreen 20 for defining the area of movement of a cursor in graphicalenvironment. This frame can also be considered a “host frame”, althoughthe interface 14 may reference it as well. In contrast, the mouse 12 hasa “local frame” 30 allowed by the workspace in which the mouse 12 ismoved. In a position control paradigm, the position (or change inposition) of a user-controlled graphical object, such as a cursor, indisplay frame 30 corresponds to a position (or change in position) ofthe mouse 12 in the local frame 28.

Rate control is also used as a control paradigm. This refers to amapping in which the displacement of the mouse 12 along one or moreprovided degrees of freedom is abstractly mapped to motion of acomputer-simulated object under control. There is not a direct physicalmapping between physical object (mouse) motion and computer objectmotion. Thus, most rate control paradigms allow the user object can beheld steady at a given position but the controlled computer object is inmotion at a commanded or given velocity, in contrast to the positioncontrol paradigm that only allows the controlled computer object to bein motion if the user object is in motion.

The mouse interface system 10 is useful for both position control(“isotonic”) tasks and rate control (“isometric”) tasks. For example, asa traditional mouse, the position of mouse 12 in the workspace 24 can bedirectly mapped to a position of a cursor on display screen 20 in aposition control paradigm. Alternatively, the displacement of mouse 12in a particular direction against an opposing output force can commandrate control tasks in an isometric mode, as described with reference tothe indexing feature of FIG. 13. Another implementation that providesboth isotonic and isometric functionality for a force feedbackcontroller and which is suitable for the interface device of the presentinvention is described in U.S. Pat. No. 5,825,308, incorporated byreference herein.

FIG. 2 is a perspective view of an example embodiment of mouse system 10with the cover portion of housing 26 removed, showing the mechanicalportion of interface 14 for providing mechanical input and output inaccordance with the present invention. A similar suitable mouse device10 for use with the present invention is described in greater detail inco-pending patent applications Ser. No. 08/881,691, filed Jun. 24, 1997,incorporated by reference herein in its entirety.

Interface 14 includes a mouse or other user manipulatable object 12, amechanical linkage 40, and a transducer system 41. A base 42 is providedto support the mechanical linkage 40 and transducer system 41 ongrounded surface 34. Mechanical linkage 40 provides support for mouse 12and couples the mouse to a grounded surface 34, such as a tabletop orother support. Linkage 40 is, in the described embodiment, a 5-member(or “5-bar”) linkage including a ground member 42, a first base member44 coupled to ground member 42, a second base member 48 coupled toground member 42, a link member 46 coupled to base member 44, and anobject member 50 coupled to link member 46, base member 48 and to mouse12.

Ground member 42 is coupled to or resting on a ground surface 34. Themembers of linkage 40 are rotatably coupled to one another through theuse of rotatable pivots or bearing assemblies (“bearings”) having one ormore bearings. Base member 44 is rotatably coupled to ground member 42by a grounded bearing 52 and can rotate about an axis A. Link member 46is rotatably coupled to base member 44 by bearing 54 and can rotateabout a floating axis B, and base member 48 is rotatably coupled toground member 42 by bearing 52 and can rotate about axis A. Objectmember 50 is rotatably coupled to base member 48 by bearing 56 and canrotate about floating axis C, and object member 50 is also rotatablycoupled to link member 46 by bearing 58 such that object member 50 andlink member 46 may rotate relative to each other about floating axis D.Mouse 12 can be moved within a planar workspace defined by the x-yplane, which is defined by the x- and y-axes as shown in FIG. 2. Mouse12 in the preferred embodiment is coupled to object member 50 by arotary bearing 60 so that the mouse may rotate about floating axis E andallow the user some flexible movement in the planar workspace.

Transducer system 41 is used to sense the position of mouse 12 in itsworkspace and to generate forces on the mouse 12. Transducer system 41preferably includes sensors 62 and actuators 64. The sensors 62collectively sense the movement of the mouse 12 in the provided degreesof freedom and send appropriate signals to the electronic portion ofinterface 14. Sensors 62, in the described embodiment, include agrounded emitter portion 70 emits a beam which is detected across a gapby a grounded detector 72. A moving encoder disk or arc 74 is providedat the end of member 48 which blocks the beam in predetermined spatialincrements. In other embodiments, other types of sensors may bepreferred, depending on the desired cost of the device. For example,instead of providing an arc 74 at the end of member 48, a number of gearteeth can be provided, which engage a cylindrical gear that is rigidlycoupled to an encoder wheel that passes through an emitter-detector pairas is well-known to those skilled in the art. Such a sensor embodimentmay provide higher sensing resolution at a lower cost, but may sufferfrom gear slip and less accurate sensing after extended use of thedevice.

Transducer system 41 also preferably includes actuators 64 to transmitforces to mouse 12 in space, i.e., in two (or more) degrees of freedomof the user object. The housing of a grounded portion of actuator 64 bis rigidly coupled to ground member 42 and a moving portion of actuator64 b (e.g., a coil) is integrated into the base member 44. The actuatortransmits rotational forces to base member 44 about axis A. The housingof the grounded portion of actuator 64 a is rigidly coupled to groundmember 42 through the grounded housing of actuator 64 b, and a movingportion (such as a wire coil) of actuator 64 a is integrated into basemember 48. Actuator 64 a transmits rotational forces to link member 48about axis A. The combination of these rotational forces about axis Aallows forces to be transmitted to mouse 12 in all directions in theplanar workspace provided by linkage 40. In the preferred embodiment,actuators 64 are electromagnetic voice coil actuators which provideforce through the interaction of a current in a magnetic field.

Another embodiment of a force feedback mouse mechanism, similar to thedescribed mechanism, is disclosed in U.S. application Ser. No.08/965,720, filed Nov. 7, 1997, and incorporated herein by reference.That embodiment includes actuators that are not stacked but are bothseparately connected to ground at two different places, and may be moreappropriate for some embodiments requiring a flatter profile for thedevice. In other interface device embodiments, other types of mechanismscan be used, such as well-known force feedback mechanisms used forjoysticks, trackballs, etc.

As shown in FIG. 3a, a workspace guide opening 76 is provided in groundmember 42 to limit the movement of mouse 12 in the x-y plane and thusdefines the limited physical workspace of the mouse 12. Guide opening 76is a shallow opening in the ground member 42 having sides which blockmovement of the mouse 12 beyond specified limits. A guide pin 78 iscoupled to the bearing 60 at axis E and extends down into the guideopening 76. Pin 78 contacts one or more sides of the opening 76 when themouse is moved to a limit in a particular direction. As shown, guideopening 76 has relatively small dimensions, allowing the mouse aworkspace of approximately 0.9″ by 0.9″ in the described embodiment;greater or smaller workspaces can be provided in alternate embodiments,and/or other types of stops or guides can be used to prevent movementpast predetermined limits. FIG. 3a shows guide pin 78 approximately inthe center of the guide opening 76. In FIG. 3b, the mouse 12 (not shown)and axis E have been moved in the x-y plane of the workspace of themouse. The movement of the mouse has been limited by the guide opening76, where guide pin 78 has engaged the sidewall of the upper-left cornerarea of guide opening 76 and stops any further movement in the forwardy-direction.

FIG. 4 is a block diagram illustrating the electronic portion ofinterface 14 and host computer 18 suitable for use with the presentinvention. Mouse interface system 10 includes a host computer 18,electronic interface 100, mechanical apparatus 102, and mouse or otheruser object 12. Electronic interface 100, mechanical apparatus 102, andmouse 12 can also collectively be considered a “force feedback interfacedevice” 104 that is coupled to the host computer. A similar system isdescribed in detail in U.S. Pat. No. 5,734,373, which is herebyincorporated by reference herein.

As explained with reference to FIG. 1, computer 18 is preferably apersonal computer, workstation, video game console, or other computingor display device. Host computer system 18 commonly includes a hostmicroprocessor 108, random access memory (RAM) 110, read-only memory(ROM) 112, input/output (I/O) electronics 114, a clock 116, a displaydevice 20, and an audio output device 118. Clock 116 is a standard clockcrystal or equivalent component used by host computer 18 to providetiming to electrical signals used by host microprocessor 108 and othercomponents of the computer system 18. Display device 20 is describedwith reference to FIG. 1. Audio output device 118, such as speakers, canbe coupled to host microprocessor 108 via amplifiers, filters, and othercircuitry well known to those skilled in the art. Other types ofperipherals can also be coupled to host processor 108, such as storagedevices (hard disk drive, CD ROM drive, floppy disk drive, etc.),printers, and other input and output devices.

Electronic interface 100 is coupled to host computer system 18 by abi-directional bus 120. The bi-directional bus sends signals in eitherdirection between host computer system 18 and the interface device 104.Bus 120 can be a serial interface bus providing data according to aserial communication protocol, a parallel bus using a parallel protocol,or other types of buses (e.g., a Universal Serial Bus (USB)).

Electronic interface 100 includes a local microprocessor 130, localclock 132, local memory 134, sensor interface 136, and actuatorinterface 138. Local microprocessor 130 preferably coupled to bus 120and is considered “local” to interface device 104, where “local” hereinrefers to processor 130 being a separate microprocessor from anyprocessors 108 in host computer 18. Microprocessor 130 can be providedwith software instructions to wait for commands or requests fromcomputer host 18, parse/decode the command or request, andhandle/control input and output signals according to the command orrequest. In addition, processor 130 preferably operates independently ofhost computer 18 by reading sensor signals and calculating appropriateforces from those sensor signals, time signals, and force processesselected in accordance with a host command, and output appropriatecontrol signals to the actuators. Such functionality is described ingreater detail in copending patent application Ser. No. 09/924,462 andU.S. Pat. No. 5,734,373, both incorporated herein by reference. Forcefeedback used in graphical environments is described in greater detailin co-pending patent application Ser. Nos. 08/571,606 and 08/756,745,which are incorporated herein by reference.

A local clock 132 can be coupled to the microprocessor 130 to providetiming data, similar to system clock 116 of host computer 18; the timingdata might be required, for example, to compute forces output byactuators 64 (e.g., forces dependent on calculated velocities or othertime dependent factors). Local memory 134, such as RAM and/or ROM, ispreferably coupled to microprocessor 130 in interface 100 to storeinstructions for microprocessor 130 and store temporary and other data.

Sensor interface 136 may optionally be included in electronic interface100 convert sensor signals to signals that can be interpreted by themicroprocessor 130 and/or host computer system 18. Actuator interface138 can be optionally connected between the actuators 64 andmicroprocessor 130 to convert signals from microprocessor 130 intosignals appropriate to drive the actuators. Power can be supplied to theactuators 64 and any other components (as required) by the USB, or by adedicated power supply 140.

Mechanical apparatus 102 is coupled to electronic interface 100preferably includes sensors 62, actuators 64, and linkage 40. Sensors 62sense the position, motion, and/or other characteristics of mouse 12along one or more degrees of freedom and provide signals tomicroprocessor 130 including information representative of thosecharacteristics. Example of sensors suitable for embodiments describedherein are rotary or linear optical encoders, potentiometers,non-contact sensors (e.g., Hall effect magnetic sensors, opticalsensors, lateral effect photo diodes), velocity sensors (e.g.,tachometers), or acceleration sensors (e.g., accelerometers).Furthermore, either relative or absolute sensors can be employed.

Actuators 64 transmit forces to mouse 12 in one or more directions alongone or more degrees of freedom in response to signals output bymicroprocessor 130 and/or host computer 18, i.e., they are “computercontrolled.” Actuators 64 can include two types: active actuators andpassive actuators. Active actuators include linear current controlmotors, stepper motors, pneumatic/hydraulic active actuators, a torquer(motor with limited angular range), a voice coil actuator, and othertypes of actuators that transmit a force to an object. Passive actuatorscan also be used for actuators 64, such as magnetic particle brakes,friction brakes, pneumatic/hydraulic passive actuators, or passivedamper elements and generate a damping resistance or friction in adegree of motion.

Mechanism 40 is preferably the five-member linkage 40 described above,but can also be one of several types of mechanisms. Other input devices141 can optionally be included in system 10 and send input signals tomicroprocessor 130 and/or host computer 18. Such input devices caninclude buttons, such as buttons 15 on mouse 12, used to supplement theinput from the user to a GUI, game, simulation, etc. Also, dials,switches, sensors, voice recognition hardware (with software implementedby host 18), or other input mechanisms can be used.

Safety or “deadman” switch 150 can be included in interface device 104to provide a mechanism to allow a user to override and deactivateactuators 64, or require a user to activate actuators 64, for safetyreasons. Safety switch 150 is coupled to actuators 64 such that the usermust continually activate or close safety switch 150 during manipulationof mouse 12 to activate the actuators 64.

Enhanced Cursor Control and Force Feedback

An aspect of the present invention is to allow control over the cursorwithout allowing limits to physical movement of the mouse (or otherobject) to become intrusive to the user. In some embodiments, the limitsare made less intrusive through the use of suitable mapping methodsbetween mouse and cursor. The limits can be made less intrusive throughthe use of “indexing” features that correspond to the case in atraditional unconstrained (open workspace) mouse of the userrepositioning the mouse in its workspace to reduce the offset betweenthe mouse frame and the host computer frame. There are several differentembodiments described herein that include these features. Although theterm “mouse” is used in the following embodiments, it is intended thatother types of interface devices and user manipulatable objects may alsobe used with the present invention, such as joysticks, finger wheels ordials, spheres, etc. In addition, the various embodiments presentedbelow are described for use with the preferred local microprocessor 130(or other dedicated processing circuitry on the interface device 104);however, a host computer 18 can implement the embodiments of the presentinvention (with any appropriate modifications) if no localmicroprocessor is present in a particular hardware embodiment.Alternatively, the host computer can implement some functions (such asballistics calculations and indexing calculations) while the localmicroprocessor implements other functions. It is assumed in the methodsbelow that host computer 18 is displaying a graphical environment suchas a GUI, game, virtual reality simulation, etc. on display device 20.It should also be noted that many of the below mapping methods can alsobe used in non-force feedback interface devices which may have physicallimits to the workspace of the manipulandum.

The methods described below may be implemented with program instructionsor code stored on or transferred through a computer readable medium.Such a computer readable medium may be digital memory chips or othermemory devices; magnetic media such as hard disk, floppy disk, or tape;or other media such as CD-ROM, DVD, PCMCIA cards, etc. The computerreadable medium may be included in the interface device 14, in hostcomputer 18, or in both. The program instructions may also betransmitted through a channel to interface device 14 from a differentsource.

Methods for performing some of the cursor control embodiments describedbelow are also described in detail in copending patent application Ser.No. 08/924,462, filed Aug. 8, 1997, entitled, “Mouse Interface Deviceand Method for Providing Enhanced Cursor Control and Indexed CursorControl with Force Feedback,” which is incorporated herein by reference.

FIG. 5 is a diagrammatic illustration of the local frame 30 and displayframe 28 and their relationship. The local frame 30 is provided in theavailable workspace in which the mouse or other user object may bemoved. In the embodiment described with reference to FIG. 2, forexample, the dimensions of the local frame 30 are defined by guideopening 76 in the base 42, which may be approximately 1″×1″ or otherdimensions. Physical limits to the local frame 30 are provided by guidepin 78 physically impacting a wall of opening 76. The mouse workspacemay be defined and limited by other mechanisms or structures in otherembodiments.

Display frame 28 is shown as a rectangle overlapping the local frame 30.Display frame 28 is the visible, displayed area on display device 20,such as the displayed portion of a video screen, on which a usercontrolled graphical object, such as cursor 180, may be moved. In FIG.5, the display frame 28 is shown as the same size as local frame 30 toemphasize certain concepts in the present invention. However, inactuality, the display frame 28 is typically larger in actual size thanthe local frame; for example, a computer monitor may have a screen of15″×11″ compared to the local frame dimensions 1″×1″. Thus, movement inlocal frame 30 is scaled up to allow movement across the entire area ofdisplay frame 28.

Local frame 30 has a local origin 182 from which x and y coordinates ofthe mouse device in its workspace are referenced. Cursor 180 is shown inFIG. 5 to represent the position of both the cursor 180 displayed indisplay frame 28 as well as the current position of the mouse 12 in thelocal frame 30 (e.g., the position of axis E and guide pin 78 in theembodiment of FIG. 2), where the tip of the cursor indicates the preciseposition. The guide pin 78 (shown as the tip of cursor 180) thus has aposition of (X_local, Y_local) in the example of FIG. 5. Likewise,display frame 28 has a screen origin 184 from which x and y coordinatesof the cursor 180 displayed on the screen 20 are referenced. The cursor180 thus has a position of (X_screen, Y_screen) in the example of FIG.5.

Border region boundaries 190 and border regions 192 of the presentinvention are illustrated in FIG. 5. Local frame 30 includes physicallimits or edges 194 which represents the physical limits to movement ofthe mouse 12 in the device workspace. For example, in the embodiment ofFIG. 2, limit 194 can be the physical walls to guide opening 76.Boundaries 190 are designated according to software (or the equivalent)by the local microprocessor 130 (or host) to be at some distance d fromthe limit 194; d can be constant around the limit 194, or d can vary atdifferent sides or portions around the workspace. The shape of theborder regions 192 can be different for each side of the workspace, ifdesired. Border region boundaries 190 define the border region 192 whichprovides a particular mouse-to-cursor mapping, as described below. Theborder region 192 borders a device interior region 193 in which adifferent mouse-to-cursor mapping is provided, as described below.Preferably, the border region 192 is an edge region that is fairly smallcompared to the size of the screen; for example, width w of the borderregion 192 can be 5% of total device workspace length or width or asimilar dimension.

In FIG. 5, the display frame 28 is shown offset from local frame 30.This can happen when the device workspace is not directly mapped tocorrespond to the screen pixel dimensions (i.e., when an absolutemapping is not used). This may cause the mouse to hit a workspace limit194 before the cursor has reached a corresponding edge of the screen. Ifa standard mouse reaches the edge of its mouse pad, it may be lifted upand re-positioned to reduce or reposition the offset between the framesand allow the mouse more workspace; thus, its workspace is effectivelyinfinite. A force feedback mouse, however, is physically constrained toa fixed-sized workspace as explained above. This workspace typicallydoes not have the same resolution as the screen the cursor moves across,and therefore this offset must be compensated for in other ways. Variousmethods used to map the physical device workspace to the screen pixelsare described below.

Cursor Control in Device Interior Region

The mouse is assumed to be in the device interior region 193 of itsworkspace most of the time, and this region is where the below-describedmethods can be applied.

Ballistics

Ballistics define a device-to-screen mapping that is dependent on thevelocity that the mouse is currently travelling at in the deviceworkspace. Ballistics helps to provide accurate control of a cursor orother graphical object when the user wishes to coarsely position thecursor, e.g., move the cursor from one object on the screen to anotheracross a large region of the screen. This type of control requires thatthe cursor be very sensitive to mouse movements so that the cursor willfly rapidly across the screen. Ballistics also helps to provide accuratecontrol of a cursor when the user wishes to finely position the cursor,e.g., to home in on a particular position, such as the interior of acheckbox or the space between two text characters. This type of controlrequires that the cursor be less sensitive to mouse movements to allowfine motions of the cursor. Often, both methods are combined in a singlemouse movement by the user: first the user swings the cursor quicklyinto a general region and then he or she homes the cursor in on thetarget, back-tracking to the target if the cursor overshoots the target.

Ballistics tries to provide accurate cursor control for the user in bothsituations. Just as an automatic transmission sets gear-ratios based onspeed, so the ballistics algorithm tries to adjust itsworkspace-to-screen ratio based on mouse speed. Herein, space in thedevice workspace is described in units of “tics”, where one ticrepresents the smallest distance (finest resolution) able to be sensedby the sensors of the device 14. For example, when using opticalencoders in a preferred embodiment, four tics are sensed when a slot ofthe encoder wheel or arc passes by the detector assembly (two detectorsare provided in quadrature sensing). Other embodiments, such as analogembodiments, can describe mouse workspace in different units ormeasurements. When the mouse is moved quickly, ballistics assumes thatcoarse positioning of the cursor is desired, so that a large number ofpixels are mapped to a particular number of tics. When the mouse ismoved more slowly, ballistics assumes that a finer cursor positioning isdesired, so that a smaller number of pixels are mapped to the samenumber of tics on the device. Embodiments using ballistics in forcefeedback mice are described in parent patent application Ser. No.08/924,462, incorporated herein by reference.

In some embodiments, ballistics methods can provide a number of discretevelocity thresholds, where cursor movement is adjusted based on theballistics relationship associated with the nearest threshold under thecurrent mouse velocity. The local microprocessor 130 or host can checkmouse velocity to determine which threshold applies. Alternatively,ballistics methods can use a continuous function to provide cursormovement based on mouse velocity. According to one ballistics method ofthe present invention, when the mouse is moved slowly (e.g. below aparticular velocity threshold), the cursor is moved based on a linearrelationship with the amount of tics the mouse is moved. When the mouseis moved quickly (e.g. greater than a particular velocity threshold),the cursor is moved based on an exponential relationship with the amountof tics the mouse is moved.

For example, in one preferred ballistics method, eight distinctthreshold relationships are provided, each relationship associated witha different velocity threshold, where three of the relationships arebased on a linear relationship with mouse motion and five of therelationships are based on an exponential relationship with mousemotion. An example of the eight thresholds and relationships is providedbelow:

Velocity Formula Relationship 0 p = c*d*k(v) p = 4d 1 p = c*d*k(v) p =5d 2 p = c*d*k(v) p = 6d 3-4 p = c*d*(d + k2(v)) p = dd 5-6 p = c*d*(d +k2(v)) p = d(d + 1) 7-8 p = c*d*(d + k2(v)) p = d(d + 2) 9-10 p =c*d*(d + k2(v)) p = d(d + 3) 11+ p = c*d*(d + k2(v)) p = d(d + 4)

The velocity is shown as ranging from 0 and up, where velocitythresholds are provided at distinct values. These values have beennormalized from actual velocity of the manipulandum as determined usingposition values. For example, the velocity can be calculated as anaverage of a number of recent velocities, where more recent velocitiescan be weighted more heavily than older velocities in the determination.Each velocity can be determined using a distance divided by timerelationship, where the distance between two tic values is used.

A formula is used to determine each relationship for each threshold. Ineach formula, p is the number of screen pixels past which the cursor ismoved, c is a scaling constant, d is the number of tics through whichthe mouse has been moved, and k(v) and k2(v) are constants that eachincrease slightly based on the velocity of the mouse as shown above. Thefirst three velocity thresholds are provided according to the formulap=c*d*k(v), where c is 1 and k(v) is either 4, 5, or 6. Thus, if themouse velocity is very slow, the first formula will be used; if themouse is slightly faster, the second formula will be used, and if themouse is moved faster than that, the third threshold is used. Theincreasing k(v) term helps to transition smoothly from the use of thefirst formula to the use of the second formula, described below.

The next five velocity thresholds are determined according to adifferent formula, p=c*d*(d+k2(v)), where c is 1 and k2(v) increases asshown. The d² relationship is provided in all five thresholds, thusproviding a much greater amount of pixels over which the cursor travelsand thus a higher cursor speed.

The ballistics method described above is ideal for force feedbackdevices (and other interface devices) having limited workspace and hardlimits or stops. The second order (d²) relationship provides cursormotion that is greatly scaled in comparison with the corresponding mousemotion, allowing the cursor to move across large areas of the screenwith very little mouse motion. In a force feedback mouse having alimited workspace as described above, such an exponential ballisticsscaling has been found to provide the user with the essential ability tomove the cursor to the edges of the screen while encountering the hardlimits to the mouse much less frequently. The second order relationshipthus works very well in conserving device workspace when manipulatingthe cursor. However, when the cursor is desired to be moved slowly, a1^(st) order relationship is used. Another advantage of the describedballistics method is that there are many more ballistics stages than intraditional ballistics methods, which yields smoother transitionsbetween the two ballistics formulas used and diminishes the feel to theuser of moving between two movement modes.

Absolute Mapping

In an absolute device-to-screen mapping there is a direct correspondencebetween the boundaries of the device workspace and the boundaries of thescreen. The tics of the device workspace are mapped directly to thescreen pixels. For example, the relationship p=c*d can be applied, wherep is the number of pixels over which the cursor is moved, c is a scalingconstant, and d is the number of tics through which the mouse has beenmoved. With the correct scaling factor, the dimensions of the workspacein tics can directly correspond to the dimensions of the screen inpixels. If the screen pixel resolution is changed, the scaling factorcan be changed to maintain the direct correspondence.

The benefit of this method is that there is no need for any borderregions 192 to control cursor movement when the mouse runs out of deviceworkspace (as described below); since with the correct scaling, thedevice need never run out of workspace and is able to control the cursorto all areas of the screen. Also, since the mapping is consistent, theuser can quickly get a sense of how far to move the mouse, whereas inballistics this is more complicated since speed of the mouse is also afactor. One drawback of this method is that cursor control is not asaccurate or convenient as in the ballistics method. At fast speeds, ittakes more mouse movement to move the cursor across the span of thescreen, while at slow speeds, fine-motion cursor control may beinadequate. Since the workspace resolution (in tics) is generally lessthan the screen resolution (in pixels), an absolute mapping may causecertain pixels on the screen may be completely inaccessible by thecursor when moved by the mouse. In some embodiments, an absolute mappingcan be used for higher mouse speeds, while a different mapping can beused at lower mouse speeds to allow access to all the pixels on thescreen, such as the linear mapping described below. Some sort ofindexing may have to be used in such an embodiment to re-center themouse in its workspace, as described below.

The borders of the absolute mapping device interior workspace may beenhanced with other control paradigms. For example, a spring force canbe allocated at the limits of the device workspace. When the user movesthe mouse against a hard limit of the workspace, a spring force opposesfurther motion into the hard limit. The spring force cushions the blowwhen the device engages the edge of the workspace.

Linear Mapping

Linear mapping defines a linear correspondence between the tics moved onthe device and the pixels on the screen. For example, a relationship ofp=c*d can be used, similar to the absolute method described above. Adifference between the linear mapping and the absolute mapping is that,when using the linear mapping, the constant c is not necessarily set togive a direct correspondence between workspace and screen. Therefore, ascaling factor can be set to provide greater cursor motion in responseto a given mouse motion, or to provide lesser cursor motion in responseto the given mouse motion. A mapping can be defined to provide finercontrol of the cursor than absolute mode. If fine cursor control isprovided, then the mouse may impact the hard limits to the deviceworkspace before the cursor reaches the edge of the screen. Therefore,other border region methods as described below can be used inconjunction with the linear mapping.

Alternatively, other methods can be used for the interior region besidesballistics to vary the scaling or the mapping of the cursor position toallow fine positioning and coarse motion of the cursor. For example, apredictive type of linear mapping (scaling) can be used, which is more“friendly” to force feedback implementations. The predictive scalingonly implements a fine-positioning scaling that is different from acoarse-movement scaling when it is deemed necessary for greater controlover the cursor, using other criteria besides mouse velocity todetermine when to alter the scaling. For example, the localmicroprocessor (or host) can examine positions of the mouse (or thecursor) over a predetermined period of time to see if a fine positioningmode is entered. The microprocessor checks whether the cursor has movedcompletely within a small region of predefined size for longer than apredetermined period of time. The region can be defined by a radius orrectangular area surrounding the mouse or cursor; for example, a regionhaving a radius of a specified fraction of screen size can be used. Thepredetermined period of time is some time period long enough to indicatethat the user is attempting to acquire a target or perform some otherfine positioning task and may be having some difficulty; for example, 3seconds can be used, or the time may depend on the particular task. Inaddition, the cursor should be in motion, since if the cursor is still,then the user may simply have taken his or her hand off the mouse, andfine positioning mode should not be entered.

Cursor Control in Device Border Region

Device border regions 192, as shown in FIG. 5, are the areas near thelimits of the device workspace. The border regions are virtual regionsin that they are designated in firmware or software and are not physicalstructures on the device. Thus, the border regions may be adjusted toany size or shape desired. The border regions can be represented as apercentage of the width of the device workspace. For example, a 10%border region size indicates a 5% left-hand border region, a 90%interior width, and a 5% right-hand border region. This percentagenumber can be similarly applied to top and bottom borders.

The use of ballistics and other variable cursor control methods causesthe mouse position in its local frame 30 to become offset from thecursor position in its display frame 28 and may eventually cause themouse to hit the workspace limits. This is simply caused by the variablescaling of cursor position based on mouse velocity used in ballistics.For example, if a mouse centered in its workspace is moved quickly tothe right by 0.5 inches from the center point, the cursor may be moved 8inches on the screen away from a screen center point. The mouse is thenmoved back the same 0.5 inches very slowly and is positioned back at theworkspace center point. However, the cursor is moved only 1 inch backtoward the screen center point due to the ballistics algorithm, creatingan offset between the mouse and cursor positions in their respectiveframes. During more movement, these offsets add up, and the mouse mayreach a physical limit to its workspace before the cursor has reached adesired target on the screen. An example of such an offset is shown inFIG. 5 as the distance between the center C_(L) of the local frame andthe center C_(S) of the screen (display frame). In such an example, themouse can hit the physical border 196 before the cursor can reach theregion 198 on the screen. Offsets in the local and display frames mayalso occur even when not using ballistics; for example, an applicationprogram or operating system may move the cursor independently of themouse, creating an offset and requiring indexing to reduce or eliminatethe offset.

Border regions can be used to reduce the problem of running out ofdevice workspace when the cursor is not yet positioned at a desiredlocation on the screen, such as at the edge of the screen. If borderregions were not used, a non-absolute interior region method (such asballistics or linear mapping) may eventually position the mouse againsta physical limit in the device workspace, where the cursor could not bemoved any further in the direction the user wishes. When the mouseenters and is positioned in a border region, a mapping method separateand distinct from the interior region mapping is applied to move thecursor on the screen. Ideally, the interior region method should beeffective enough so that the mouse is moved to the border regions veryinfrequently.

Rate Control

As described above, rate control is a mapping in which the displacementof the mouse 12 is abstractly mapped to motion of a computer-simulatedobject under control, not directly mapped. Thus, the mouse need not bein motion for the cursor to be in motion. In the preferred embodiment,rate control is implemented such that the position of the mousedetermines the velocity of the cursor instead of the position of thecursor. Rate control is provided when the mouse is moved from theinterior region to a border region. Once the mouse enters the borderregion, the cursor continues moving toward the closest edge of thescreen while the mouse is preferably impeded in the correspondingdirection by a resistive force, such as a spring force, damping force,frictional force, etc. A preferred embodiment uses a spring force. Forexample, moving the mouse into the left border region causes the cursorto continue moving toward the left edge of the screen, while the mouseis impeded in the left direction by a spring force in the rightdirection (the mouse can preferably be moved to the right freely, and aninterior region method can be used immediately upon movement to theright). The cursor moves towards the edge of the screen based on thedistance d that the mouse is penetrating into the border region. Thisdistance can be expressed as a ratio r=d/b, where d is the distance ofpenetration and b is the total width of the border region. A function isthen applied to d to achieve the desired cursor velocity v=f(d).

The function f(d) can vary depending on the control scheme. In “dualaxis” rate control (see below), the cursor is moved on the screen in twodegrees of freedom using the rate control method while the mouse is inthe border region. The function applied to d for such a control schemecan be linear, i.e. v=cd where c is a constant scaling factor. In termsof pixels on a screen, the relationship is p=r*c, where p is the numberof pixels the cursor is moved since the last displayed position and r isthe ratio described above. The velocity of the cursor thus will belinearly based on the distance d penetrated into the border region. Thisinsures that the cursor moves in the direction the user is pushing themouse. This linear function also works well for “single axis” ratecontrol (see below), where only one mouse direction or degree of freedomuses the rate control method. It is also possible to use a differentfunction that follows a more complicated profile. For example, afunction might provide a “deadband” region when the border region isfirst entered, where the cursor speed does not increase as thepenetration into the border region increases. The deadband region wouldbe followed by a region where the cursor movement increases slowly aspenetration distance increases. Once the mouse has penetrated halfwayinto the border region, the velocity of the cursor can increaseexponentially in relation to d. Alternatively, a ratio r can bedetermined based on a minimum width b of the border region, where if bis greater than a particular threshold value, it is assigned thatthreshold value. This allows the cursor to achieve a maximum velocityafter the mouse is moved a desired threshold distance into the borderregion, regardless of how large b is and without the user having to movethe mouse all the way to the edge of the workspace to cause the fastestcursor speed.

As mentioned above, a virtual spring preferably extends across ratecontrol border regions such that a spring force opposes the movement ofthe mouse into the region. The spring force is strongest at the edge ofthe device workspace, so that it is most difficult to move the mousetoward a workspace edge when the mouse is close to the edge. The springis force feedback well-suited for rate control as it creates acorrelation that the user can intuitively understand between thepressure the user exerts and the rate of the cursor. If the user easesoff of the mouse, the spring will push the mouse out of the boundary.This makes it easy to stop the cursor from moving.

Single Axis Rate Control

Single axis rate control provides a rate control scheme only for onedegree of freedom (or direction) of the mouse. The assumption thismethod makes is that rate control will only be used in the direction inwhich workspace needs to be conserved, e.g. if the mouse is at a leftworkspace limit, the mouse still has room to move up, down, and right.Preferably, single axis rate control affects the degree of freedomrelevant to the border region. For example, if the mouse enters the leftborder region, a spring is felt by the user on the left side of themouse which controls the cursor speed in the left direction on theX-axis according to the rate control method; the left-right degree offreedom on the mouse is the relevant degree of freedom to the borderregion. However, if the mouse is moved up or down within the borderregion, the interior region mapping method is used for cursor movementalong the Y-axis. Rate control of the cursor is only provided along oneaxis unless the mouse happens to be in two border regionssimultaneously, such as at a corner of the device workspace at theintersection of two border regions, e.g. region 195 in FIG. 5, whererate control can be provided in both degrees of freedom of the mouse.

Dual Axis Rate Control

In contrast to single axis rate control, dual axis rate control (or“radial rate control”) affects both axes simultaneously. In other words,when the mouse is moved into a border region, a mode is entered wherethe cursor position in both the X and Y axes is calculated using a ratecontrol method. In the case where the mouse has entered the leftboundary region, rate control and a spring resistance is provided at theleft (but not to the right, as described below). In addition, ratecontrol is provided in the forward-back degree of freedom and Y axis.The Y-position of the cursor upon entry to the border region is used asthe reference or “zero” point, where there is no Y-axis movement. If theuser moves the mouse above this reference point, the cursor will moveupwards according to a rate control method. If the user moves the mousebelow the reference point, the cursor will move downwards according tothe rate control scheme. Rate control mode can be exited by moving themouse to the right, away from the left border region. Preferably, theuser does not also have to return the mouse/cursor to the referenceY-position to exit rate control mode, but such an embodiment canalternatively be implemented.

The assumption this method makes is that rate control is fundamentallydifferent from other position control methods and it is disconcerting toa user to be in rate control in one axis and a different paradigm suchas position control in the other axis. Dual axis rate control keeps themouse in a single mode, either totally in rate control in both degreesof freedom or totally in position control in both degrees of freedom.

The rate control in the degree of freedom of the mouse not relevant tothe border region (the “non-relevant degree of freedom” being the degreeof freedom not close to an edge of the workspace, e.g., the Y-axis inthe above example) can be implemented in different ways. In a preferredembodiment, no forces are provided in the non-relevant degree offreedom. Thus, in the example above, there would be a spring on theleft, but no up or down spring. This indicates that the axes are stillacting differently since the mouse still has available workspace in oneaxis but not in the other. In other embodiments, a spring force (orother type of resistive force) can be provided in the non-relevantdegree of freedom as well as in the relevant degree of freedom.

Variable Absolute Control

The variable absolute mapping method maps the distance between thecursor and the nearest edge of the screen to the remaining distancebetween the mouse and the edge of the device workspace (also called theedge-scaling method in parent application Ser. No. 08/924,462). When themouse reaches the border region, the variable absolute mapping is thenapplied to provide the proper scaling to the remaining distance to theworkspace limit. Thus, if the mouse reaches a workspace limit, thecursor will have always moved fully to the edge of the screen. If theuser moves the mouse half of the distance to the workspace edge, thecursor is moved half the distance to the edge of the screen, regardlessof how many screen pixels are actually traversed. This guarantees thatthe cursor will hit the edge the same time the workspace is exhausted.It is similar to absolute mapping or linear mapping in that the mappingis linear, except that the constant multiplying factor is determinedwhen the border region is entered.

Once the mouse enters the border region of the workspace, variableabsolute mode is in effect while the cursor is travelling towards theedge of the screen. However, if the mouse is moved away from the closestworkspace limit to move the cursor in the opposite direction, deviceworkspace towards the near edge no longer needs to be conserved, and thenormal interior range mapping method can be used instead of the variableabsolute mapping method, even if the mouse is still located within theborder region.

Since variable absolute mapping is a position control mapping, i.e., thedevice position still corresponds to the cursor position, transitionfrom a position-control interior region to a variable absolute borderregion has a much less “modal” feel than transition from aposition-control interior region into a rate control border, wheredevice position corresponds to velocity.

Variable absolute mapping may cause some problems for the user with finepositioning of the cursor within the edge-scaled region of the screen,since the cursor motion is scaled higher in this region. However, theedge scaling is used only in the direction towards the edge of thescreen. Thus, if the user overshoots a target during the edge scaling,the user may move the mouse in the opposite direction to acquire thetarget, at which point an interior region mapping is used whichtypically allows easier fine positioning.

In other embodiments, the local microprocessor or host computer can becontinually scaling at least one area of the mouse and screen accordingto the variable absolute mapping method regardless of cursor positionand not only when the cursor is in a border region. A new scaling factorcan be calculated in real time for all positions of the mouse in itsworkspace, not just for regions close to the edge of the workspace. Forexample, the area of the workspace between the mouse and the closestlimit can be scaled. The microprocessor would always be examining thedistance between the current mouse position and the workspace limit andthe distance between the cursor and the screen limits and scaling thecursor position accordingly. In one example, three “cursor speeds”(i.e., cursor scalings) can be provided: coarse, fine, and intermediate.Coarse and fine speeds are constant mappings of cursor to mouse positionallowing different degrees of control. However, the intermediate speedcan vary the scaling factor according to the offset between local anddisplay frames. In an alternative embodiment, the microprocessor candetermine the distance of the mouse and cursor to limits on all sides,such that four different scaling factors can be stored and the one thatcorresponds to the cursor's actual direction is used.

A damping force can be applied inside variable absolute border regions.The damping force slows the mouse down, cushions impact at the edge ofthe device workspace, and provides feedback to the user that the mouseis near the edge of the device workspace.

Hybrid Methods

Hybrid mapping methods choose an appropriate mapping method that allowsa high velocity of the cursor to be maintained when the cursor movesfrom an interior region to a border region (or vice versa in alternateembodiments). The mapping method providing the greater cursor speed isselected and its result used for repositioning the cursor. For example,the mouse may be moved in an interior region towards a border region,which moves the cursor toward the edge of the screen. When the mouse isin the border region and moved towards the workspace limit,corresponding pixel offsets according to both the interior regionmapping method and the border region mapping method are calculated orotherwise determined (e.g. by the local microprocessor 130 or host). Thetwo pixel offsets are compared and the larger of the two offsets is usedto display the cursor at a new location on the screen. This effectivelymoves the cursor at the fastest rate provided by the two mappingmethods. The offsets from both mapping methods can continue to becompared at regular intervals while the mouse is moved within the borderregion toward the workspace edge, and the fastest offset taken at eachinterval. Alternatively, the comparison can be made only once when themouse first enters the border region (or re-enters the border region).

Since border region mapping methods are used to conserve deviceworkspace when the mouse is close to a limit, it makes little sense totransition to a border region method if the border region methodactually makes poorer use of remaining device workspace. For example, ifan interior region method moves a cursor at a particular velocity, andthe border region method actually moves the cursor at a slower velocity,the mouse may hit the workspace limit before the cursor hits the edge ofthe screen. In most situations, a cursor should not be slowed down whenthe mouse is about to hit a border region. Hybrid methods most oftencome into play when the user is moving the mouse quickly using aninterior ballistics algorithm.

A preferred embodiment of the hybrid mapping method 200 is summarized inthe flow diagram of FIG. 6. The method starts at 202, and in step 204,the border region and interior region of the device workspace isdefined. This can be defined based on user preferences, or defaultvalues. For example, as explained above, a 5% border on each side of theworkspace can be defined, with the interior region defined as theremaining area.

In step 206, the process checks whether the mouse (or other manipulandumfor a particular device) is crossing from the interior region to theborder region. For example, a history of a predetermined number ofprevious position values of the mouse can be examined to determine ifthe boundary between regions has been recently crossed. If this is notthe case, then the mouse is moving within either the interior region orwithin the border region (or is moving from the border region to theinterior region), and step 208 is initiated. In step 208, the processchecks whether the mouse is currently located in the interior region. Ifso, then in step 210, the process applies an interior mapping, such as aballistic mapping or other mapping described above. The process thenreturns to step 206.

If the mouse is not currently in the interior region in step 208, thenthe mouse must be currently located in the border region, and step 212is initiated. In step 212, the process applies a border mapping to thecontrol of the cursor, For example, the rate control method describedabove can be provided in the direction toward the workspace limit toallow rate control over the cursor. The process then returns to step206.

If in step 206 the mouse is detected to have crossed from the interiorregion to the border region, then in step 214 the process selectsbetween the interior mapping and the border mapping to control thecursor. The mapping which causes faster cursor motion is the mappingthat is selected, as described above. For example, the processor or hostcan calculate the pixel offset for each of the mappings and choose themapping providing the greater pixel offset for the cursor. In step 216,the selected mapping is applied to move the cursor on the screen. Theprocess then returns to step 206. It should be noted that, even if theinterior mapping is selected to be applied in step 216, any borderforces associated with the border mapping are still preferably applied,such as opposing spring forces, to provide consistency. Such opposingforces also tend to cushion the mouse against hitting a hard stop suchas the edge of the workspace.

As mentioned above, in alternate embodiments, the selection betweenmappings can be made each time the mouse is found to be currentlylocated in the border region, rather than only when the mouse firstcrosses into the border region.

The hybrid method has the effect of making smoother transitions from aninterior region method into a border region method. If the user ismoving the cursor rapidly, the cursor continues to be moved rapidlyafter the mouse enters the border region and the user cannot visuallydetect a difference in mapping regions. While the mouse is in the borderregion and moving toward the edge, the interior region method offset mayfluctuate as the mouse is slowed down and the border region method cantake over when it would provide faster cursor movement; and vice versa.In addition, the hybrid method typically has the effect of synchronizingthe local and host frames at a faster rate. Thus, if the deviceworkspace limit is reached by the mouse, the cursor is moved as fast aspossible to the edge of the screen so that the cursor position at theedge of the screen is synchronized with the mouse position at the edgeof the workspace.

Other Embodiments

As explained above and in U.S. Pat. No. 5,825,308 and patent applicationSer. No. 08/924,462, both incorporated herein by reference, indexing isused to change the offset between the local frame and the host frame.Indexing with a traditional, unconstrained mouse is accomplished byphysically lifting the mouse and moving it to a different location,where input signals are not sent to the host computer while the mouse ismoved. A force feedback mouse as described herein typically cannot bephysically lifted from its linkage during use, but the temporarysevering of the connection between mouse and host computer can beachieved through an alternate input device, such as the depressing of abutton on the mouse, the activation of a hand-weight switch in themouse, or other activation of an input device. When such an input deviceis pressed, the mouse is virtually “lifted,” achieving the samefunctional indexing mode as lifting a regular mouse.

In some embodiments, the lifting motion of a regular mouse can beapproximated or simulated in a grounded force feedback mouse by sensingactual upward (z-axis) motion of the mouse out of its plane of motionwhen the mouse is lifted by the user. For example, physical play can beallowed in the mouse mechanism along the z-axis, e.g., a hinge, flexurejoint, or other mechanism can be provided to allow the mouse to belifted or pivoted upward by a predetermined amount (before hitting astop or other limit), or pressure of the mouse in the z-axis can besensed. A sensor or switch can detect such movement to indicate indexingmode is active. This allows the user to provide indexing with the forcefeedback mouse using a similar method to that of a traditional,unconstrained mouse. For example, the ground member 42 shown in FIG. 2can be rotatably coupled to the surface 34 so that the entire mouseassembly can be rotated about axis H, allowing the user to pivot themouse upwards. Alternatively, just the mouse portion 12 can be allowedto rotate upward with respect to the linkage 40 and other parts of themechanism.

In some indexing embodiments, a spring force or other centering forcecan be temporarily applied to the planar degrees of freedom of the mouseusing the actuators of the mouse device while indexing mode is active.The centering force guides the mouse to a position in the deviceworkspace which provides a more even distribution of distance from themouse to the edges of the workspace, thus helping to prevent the mousefrom being too close to a workspace limit (an “auto-centering”embodiment). For example, the centering force can guide the mouse to themidpoint between the center of the device workspace and the location inthe device workspace where the mouse would be if the mouse were in anabsolute mapping mode (as described above) given the current cursorposition. In other words, a compromise is made between centering themouse in the workspace and placing it in a position that relates to thecurrent cursor position. The mouse can be guided to other locations inother embodiments, such as the center of the workspace, or to apredetermined distance from the nearest workspace limit.

Auto centering can also be performed only when the user is not graspingthe mouse and/or using the mouse for cursor positioning tasks, since theauto-centering movement might confuse the user. In addition, the autocentering can be performed only when the offset between frames increasesover a predetermined threshold. Alternatively, a special button, switch,or other input device can be provided to the user on mouse 12 or otherposition which would cause the mouse to be auto centered when the inputdevice is selected by the user. An application program can also commandan autocentering function to take place using a host command sent to theforce feedback device.

No border region methods are theoretically needed if an indexing abilityfor the force feedback mouse is provided. However, in many practicalembodiments, the workspace resolution is small enough such that, apartfrom absolute mapping mode, border regions are often still desirable.

Software Interface

Many of the features disclosed herein can be presented for a user in asoftware interface. Since many of the different mapping methods can bevaried to provide different cursor movement and since different usersmay prefer different methods, the user should be presented with an easyway to select between various mapping methods. A software control panelcan be used to present various options to the user.

FIG. 7 illustrates two such software interfaces which allow the user toselect between different mapping methods. Interface window 300 can bedisplayed by an application program or through a system program in aGUI. A user can select a sensitivity value 302 for the mouse-to-cursormapping, which can adjust the scaling factor of the mapping used. Aglobal damping control 304 can be used to select whether any globaldamping force is provided. As described in copending patent applicationSer. No. 08/924,462, a resistive damping force can be used to slow downthe cursor to allow better cursor control. “Normal Damping” can providea global damping force that increases the damping resistance with thevelocity of the mouse. “Envelope damping” can provide a morevelocity-dependent damping, e.g. greater damping at slow mouse speeds toprovide more accurate control, and less damping at faster mouse speedsto allow easier control over large motions of the cursor. Damping forcescan be provided based on other conditions as well, such as velocitythresholds similar to ballistics embodiments.

The Device Interior control 306 allows the user to select the mappingmethod used for the interior region of the mouse workspace, whereballistics, linear, and absolute mappings are listed. The Device Bordercontrol 308 allows the user to select characteristics for the borderregion. For example, the user can select the size of the border region,which also governs how large the interior region will be. The maximumborder region size is the entire screen, such that no interior region ispresent. The user can also select the strength (magnitude) of forceopposing entry into the border region. The user can select single-axisrate control, dual axis rate control, or variable absolute mappingmethods for the border region. Finally, the user can select whetherhybrid (velocity matching) methods are used upon entry to the borderregion.

Interface window 320 presents a more simplified interface which presentsfewer options to the user, and which may be more suitable for beginningusers. The user can adjust basic characteristics such as speed 322(scaling), device interior region characteristics 324, and device borderregion characteristics 326. Less options are provided in each selectionarea.

While this invention has been described in terms of several preferredembodiments, it is contemplated that alterations, permutations andequivalents thereof will become apparent to those skilled in the artupon a reading of the specification and study of the drawings. Forexample, although examples in a GUI are described, the embodimentsherein are also very well suited for other two-dimensional graphicalenvironments and especially three-dimensional graphical environments,where a user would like fine positioning in manipulating 3-D objects andmoving in a 3-D space. For example, the rate control regions are quitehelpful to move a cursor or controlled object in a 3-D environmentfurther than physical limits of the interface device allow. In addition,many different types of forces can be applied to the user object 12 inaccordance with different graphical objects or regions appearing on thecomputer's display screen and which may be mouse-based force sensationsor cursor-based force sensations. Also, the various features of theembodiments herein can be combined in various ways to provide additionalembodiments of the present invention. In addition, many types of userobjects and mechanisms can be provided to transmit the forces to theuser, such as a mouse, trackball, joystick, stylus, or other objects.Furthermore, certain terminology has been used for the purposes ofdescriptive clarity, and not to limit the present invention. It istherefore intended that the following appended claims include all suchalterations, permutations, and equivalents as fall within the truespirit and scope of the present invention.

What is claimed is:
 1. A tactile feedback interface device providingcursor control, said interface device in communication with a hostcomputer that displays graphical objects in a graphical environment on adisplay screen, said interface device comprising: a manipulandum movablein a physical workspace that controls a position of a cursor displayedby said host computer within said graphical environment, said physicalworkspace having limits to motion of said manipulandum; at least onesensor operative to detect motion of said manipulandum and provide asensor signal, wherein sensor data based on said sensor signal isreported to said host computer to allow positioning of said cursor,wherein a border region and an interior region are defined in saidphysical workspace of said interface device, and wherein aposition-control mapping of movement of said manipulandum is applied tomovement of said cursor when said manipulandum is positioned in saidinterior region, and a rate control mapping of movement of saidmanipulandum is applied to movement of said cursor when saidmanipulandum is positioned in said border region; and at least oneactuator operative to output a tactile sensation to a user.
 2. A tactilefeedback interface device as recited in claim 1 wherein said ratecontrol mapping uses penetration of said manipulandum into said borderregion to control a speed of movement of said cursor along a particulardirection on said display screen.
 3. A tactile feedback interface deviceas recited in claim 2 wherein said tactile sensation output to said useris a resistive force on said manipulandum resisting penetration of saidmanipulandum into said border region from said interior region, amagnitude of said force being based on the depth of said penetrationinto said border region.
 4. A tactile feedback interface device asrecited in claim 1 further comprising a processor operative to providesignals to said at least one actuator to control said output of saidtactile sensation to said user.
 5. A tactile feedback interface deviceas recited in claim 1 wherein said position control mapping is aballistics mapping, wherein a speed of said cursor in said graphicalenvironment depends on a speed of said manipulandum in said physicalworkspace.
 6. A method of providing control of a cursor displayed on ascreen using a tactile feedback interface, said method comprising:providing a user manipulatable object, said object moveable by a user inat least one degree of freedom in a workspace; providing at least onesensor for detecting and reporting the motion of said user manipulatableobject in said at least one degree of freedom; enabling control of agraphical cursor on said screen, said cursor being controllable throughtwo different modes, a position control mode and a rate control mode,said position control mode mapping displacement of said usermanipulatable object to displacement of said cursor, said rate controlmode mapping displacement of said user manipulatable object to a rate ofdisplacement of said cursor, wherein said position control mode is usedwhen said user manipulatable object is in a predefined center region ofsaid workspace and wherein said rate control mode is used when said usermanipulatable object is at or outside borders of said predefined centerregion of said workspace; and enabling the output of tactile sensationsto said user through said tactile feedback interface, said tactilesensations corresponding with said cursor moving between graphicalelements displayed on said screen, wherein said tactile sensations arebased on a position of said user manipulatable object when said positioncontrol mode is in use, and said tactile sensations include sensationsbased on time when a rate control mode is in use.
 7. A method as recitedin claim 6 wherein said graphical elements include menu elements in amenu, and wherein said tactile sensations indicate when said cursormoves from one menu element to another menu element in said menu.
 8. Amethod as recited in claim 7 wherein said position-based tactilesensations are detent sensations and wherein said time-based tactilesensation are vibration sensations.
 9. A method as recited in claim 6wherein said tactile sensations output when said rate control mode is inuse include a resistive spring force in addition to said time basedtactile sensations, wherein a magnitude of said resistive spring forceis based on said distance of said mouse past a border of said borderregion.
 10. A method as recited in claim 9 wherein said time basedtactile sensations are output synchronized with cursor interactions withgraphical elements displayed on said screen.
 11. A method as recited inclaim 6 wherein said position control mode provides ballistic control ofsaid cursor.
 12. A method for providing cursor control on a cursorcontrol interface device, said interface device coupled to a hostcomputer that displays graphical objects in a graphical environment on adisplay screen, said interface device including a manipulandum movablein a physical workspace in order to control the position of a cursordisplayed by said host computer within said graphical environment, themethod comprising, enabling an interior mapping of movement of saidmanipulandum to be applied to movement of said cursor when saidmanipulandum is positioned in an interior region of said physicalworkspace; enabling a border mapping of movement of said manipulandum tobe applied to movement of said cursor when said manipulandum ispositioned in a border region adjacent to said interior region; andenabling a selected one of said interior mapping and said border mappingof movement of said manipulandum to be applied to movement of saidcursor when said manipulandum crosses from said interior region to saidborder region, wherein said selected mapping causes a greater velocityof said cursor than the unselected mapping.
 13. A method as recited inclaim 12 wherein said border region is adjacent to at least one limit ofsaid physical workspace.
 14. A method as recited in claim 12 furthercomprising outputting a force on said manipulandum when saidmanipulandum is in said border region.
 15. A method as recited in claim12 further comprising outputting a force on said manipulandum resistingsaid penetration into said border region, a magnitude of said forcebeing based on the depth of said penetration into said border region.16. A method as recited in claim 15 wherein said border mapping is arate control mapping wherein said penetration into said border region isused to control a speed of movement of said cursor along a particulardirection on said display screen.
 17. A method as recited in claim 12wherein said interior mapping is a ballistics mapping wherein a speed ofsaid cursor depend on speed of said manipulandum.
 18. A method asrecited in claim 15 wherein said force resists motion of said mouse onlytoward said physical limit adjacent to said border region.
 19. A methodas recited in claim 12 further comprising determining an amount ofscreen pixels that each of said mappings would cause said cursor to movepast, and applying said mapping that causes said cursor to move past thegreater number of screen pixels.
 20. A force feedback device providingautomatic centering, said device in communication with a host computerthat displays graphical objects in a graphical environment on a displayscreen, said device comprising: a manipulandum movable in at least onedegree of freedom in a local frame, wherein said manipulandum hasphysical limits to said movement in said local frame; at least oneposition sensor detecting motion of said manipulandum and providing asensor signal, wherein sensor data based on said sensor signal isprovided to said host computer to allow positioning of said cursor in ascreen frame; and at least one actuator outputting a force on saidmanipulandum in at least one degree of freedom of said manipulandum andcausing said manipulandum to move in said local frame to decrease anoffset between said local frame and said screen frame.
 21. A forcefeedback device as recited in claim 20 wherein said actuatoradditionally outputs forces coordinated with the display of imagesdisplayed by said host computer.
 22. A force feedback device as recitedin claim 20 further comprising a processor operative to provide signalsto said actuator to control said force to move said manipulandum in saidlocal frame.
 23. A force feedback device as recited in claim 20 furthercomprising a contact sensor operative to determine when said user iscontacting said manipulandum, wherein when said user is not contactingsaid manipulandum, said forces to move said manipulandum in said localframe to decrease said offset between said local frame and said screenframe.
 24. A force feedback device as recited in claim 20 wherein saidmanipulandum is moved with said forces to a location at a predetermineddistance from a nearest workspace limit.
 25. A force feedback device asrecited in claim 20 wherein said manipulandum is moved decrease saidoffset between said local frame and said screen frame after said offsetincreases over a predetermined threshold offset after use of said forcefeedback device.